Papers with Chain-of-Thought and Program-of-Thought
KnowledgeFMath: A Knowledge-Intensive Math Reasoning Dataset in Finance Domains (2024.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for large language models (LLMs) are only 56.6% accurate, leaving room for improvement. |
| Approach: | They propose a benchmark to evaluate LLMs' capabilities in solving knowledge-intensive math reasoning problems using a finance-domain knowledge bank and expert-annotated solution references. |
| Outcome: | The proposed system achieves only 56.6% accuracy, leaving room for improvement. |